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Overcoming  the  curse  of  dimension:  Methods 
based  on  sparse  representation  and  adaptive 

sampling 

Weinan  E 

Department  of  Mathematics,  Princeton  University 


1  Introduction  and  motivation 

A  major  issue  in  modeling  and  computation  is  how  to  handle  high  dimensional  problems. 
We  can  divide  these  high  dimensional  problems  into  two  classes:  Moderately  high  dimen¬ 
sional  problems  or  very  high  dimensional  problems.  In  the  former  class,  we  have  problems 
such  as  the  Boltzmann  equation,  whose  dimensionality  is  high  but  they  are  still  amend¬ 
able  to  grid-based  methods.  In  the  latter  class  we  have  problems  such  as  exploration  of 
the  configuration  space  of  a  large  molecule.  These  problems  often  involve  hundreds  of 
thousands  of  dimensions,  and  methods  based  on  fixed  grids  are  far  from  being  adequate. 
We  have  explored  various  ways  of  handling  these  problems  using  the  sparse  representation 
or  the  adaptive  sampling. 

2  Solving  Boltzmann  and  Fokker-Planck  equations 

Boltzmann  equation  is  one  of  the  most  fundamental  problems  in  physics.  Fokker-Planck 
equations  have  also  received  a  great  deal  of  interest  in  recent  years,  particularly  in  the 
chemical  engineering  community,  in  the  context  of  studying  complex  fluids. 

From  a  numerical  viewpoint,  a  common  difficulty  is  that  dimensionality.  For  example, 
Boltzmann  equation  is  usually  a  7  dimensional  problem,  6  space  dimensions  plus  time. 
Therefore  traditional  grid-based  methods  are  ineffective.  The  method  of  choice  has  been 
Monte  Carlo  method. 

Monte  Carlo  method  suffers  from  two  difficulties.  One  is  the  slow  convergence  rate. 
The  other  is  the  numerical  noise  in  the  solutions.  For  this  reason,  it  is  still  highly  de¬ 
sirable  to  develop  effective  grid-based  methods  that  solve  directly  the  partial  differential 
equations. 

In  collaboration  with  Jie  Shen,  we  have  formulated  such  a  method  using  sparse  rep¬ 
resentation,  including  sparse  grids  and  sparse  spectral  methods.  The  method  is  quite 
promising  in  the  sense  that  it  provides  an  effective  technology  for  solving  intermediate 
dimensional  partial  differential  equations. 
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Since  the  work  falls  mainly  in  the  domain  of  expertises  of  Jie  Shen,  the  subsequent 
program  has  been  carried  out  mainly  by  him,  together  with  our  joint  post-doc  Haijun  Yu. 
Please  refer  to  his  report  for  the  progress  made  in  this  direction. 

3  Exploring  parameter  space  using  sparse  representation 

3.1  Calibrating  and  improving  empirical  inter-atomic  potentials 

One  of  the  most  important  issue  in  atomistic  modeling  is  to  determine  the  inter-atomic 
potential.  The  process  of  that  often  works  as  follows.  One  first  specifies  a  form  of  the 
potential,  based  on  experience  or  known  facts.  This  often  involves  many  undetermined 
parameters.  One  then  determines  the  value  of  these  parameters  using  experimental  data 
or  data  obtained  from  first  principle  calculations.  Needless  to  say,  the  success  of  such 
empirical  potentials  depends  heavily  on  the  assumed  form. 

Assess  the  accuracy  of  such  an  empirical  potential  is  a  rather  difficult  task  since  the 
dimension  of  the  configuration  space  of  the  atoms  is  usually  quite  huge.  One  question 
the  PI  has  pursued,  in  collaboration  with  physicists  Gang  Lu  at  the  California  State 
University  at  Northridge,  is  to  assess  the  validity  of  the  embedded  atom  model  in  the 
elastic  regime.  The  embedded  atom  model  is  a  very  popular  model  for  studying  metallic 
systems.  The  space  of  elastic  deformations  is  parametrized  by  the  strain,  which  is  a 
6  dimensional  space.  Using  sparse  representation  in  6  dimensional  space,  the  PI  and 
collaborators  were  able  to  systematically  explore  the  space  of  elastic  deformation.  It  was 
found  that  the  embedded  atom  model  works  quite  well  when  the  system  is  under  shear  or 
tension,  but  it  works  quite  poorly  when  the  system  is  under  compression.  In  analogy  with 
the  modeling  of  exchange-correlation  functionals  in  density  functional  theory  and  viewing 
the  embedded  atom  model  as  the  analog  of  the  local  density  approximation  (LDA),  the 
PI  and  co-workers  have  developed  an  analog  of  the  generalized  gradient  approximation 
(GGA).  Current  results  suggest  that  this  improved  model  gives  much  better  results  in  all 
regimes.  The  results  are  published  in  [1]. 

3.2  Sequential  multiscale  modeling 

Sequential  multiscale  modeling  has  certain  advantages.  For  example,  it  does  give  us  a 
model  to  work  with,  not  just  for  computational  purposes,  but  also  for  analytical  purposes. 
On  the  other  hand,  it  if  often  believed  that  suquential  multiscale  modeling  is  limited 
to  the  passage  of  a  few  parameters  from  microscale  models  to  macroscale  models.  If  the 
unknown  component  of  the  macroscale  model  is  a  function  that  depends  on  many  variables, 
sequential  multiscale  modeling  stops  being  effective.  This  is  generally  true.  However,  the 
power  of  sequential  multiscale  modeling  can  be  greatly  improved  if  sparse  representations 
are  used.  For  example,  assume  that  we  are  modeling  complex  materials  or  complex  fluids, 
for  which  the  constitutive  law  for  the  stress  is  either  a  function  of  the  strain  or  the  rate 
of  strain,  which  in  three  dimension  is  a  function  of  6  variables,  then  it  becomes  feasible  to 


2 


precompute  the  constitutive  information  using  sparse  representation.  This  is  illustrated 
in  [2], 

This  has  important  consequences  in  multiscale  modeling:  In  many  cases,  it  is  enough 
to  use  sequential  multiscale  modeling  if  we  represent  the  constitutive  information  in  a 
smart  way. 

4  Exploring  very  large  dimensional  configuration  space  us¬ 
ing  adaptive  sampling 

Sparse  representation  is  only  effective  if  the  dimension  of  the  problem  is  only  moderately 
high.  In  computational  science  we  often  encounter  problems  whose  dimension  can  be 
thousands  or  even  larger.  In  this  case,  we  have  to  use  other  strategies. 

One  strategy  that  we  have  pursued  so  far  is  adaptive  sampling.  The  general  procedure 
is  as  follows.  We  first  identify  a  lower  dimensional  region  (e.g.  points,  curves  or  surfaces) 
in  the  configuration  spaces.  We  then  sampling  the  probablity  distribution  of  our  interest  in 
the  neighborhood  of  that  region,  and  determine  how  the  region  moves  in  the  iteration.  Such 
an  adaptive  sampling  procedure  has  been  used  with  great  success  to  gradient  systems,  i.e. 
systems  that  have  an  underlying  energy  landscape,  with  applications  to  many  important 
problems  in  material  science,  chemistry  and  biology. 

Our  new  interest  is  to  study  non-gradient  systems,  of  which  fluid  dynamics  is  a  good 
example.  For  that  purpose,  we  will  proceed  in  a  number  of  steps.  We  will  use  transition 
pathways  between  stable  states  as  an  example. 

4.1  The  adaptive  minimum  action  method 

Given  two  stable  or  metastable  states,  we  would  like  to  find  the  most  probable  transition 
path  between  the  two  states.  For  gradient  systems,  the  string  methods  have  been  very 
successful.  For  non-gradient  systems,  we  are  exploring  the  mimimum  action  method,  which 
is  more  general,  but  to  be  effective,  it  requires  a  high  quality  mesh  along  the  path  in  order 
to  resolve  the  path.  This  is  very  important  since  these  paths  are  often  very  complicated. 
The  PI  and  collaborators  have  developed  an  adaptive  minimum  action  method.  The  basic 
idea  comes  from  the  moving  mesh  method.  The  objective  is  to  find  the  optimal  mesh 
using  carefully  chosen  monitor  functions,  as  the  iteration  proceeds.  This  is  very  simple, 
but  it  proves  to  be  quite  effective  [3] . 

4.2  Application  to  the  KS  equation 

As  an  application,  the  PI  and  co-workers  have  used  the  adaptive  minimum  action  method 
to  study  the  dynamics  of  the  Kuramoto-Sivashinsky  equation  in  its  configuration  space. 
One  first  identifies  a  stable  stationary  solution  and  another  traveling  wave  solution.  By 
finding  transition  pathways  between  these  two  solutions,  one  further  finds  more  stationary 
solutions.  More  importantly,  one  can  further  refine  the  whole  procedure  and  find  impor¬ 
tant  objects  on  the  separatrices  between  the  basins  of  attractions  of  the  different  stable 
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solutions.  This  procedure  gives  us  a  rich  set  of  information  about  the  very  complicated 
configuration  space  of  the  Kuramoto-Sivashinsky  equation.  The  results  are  documented 
in  [4], 

The  example  studied  so  far  is  the  Kuramoto-Sivashinsky  equation.  But  the  method¬ 
ology  is  fairly  general. 

We  are  in  the  process  of  applying  this  methodology  to  study  noise-induced  instability 
and  transition  to  turbulence  in  incompressible  fluids.  We  have  made  some  initial  progress, 
but  this  is  a  very  difficult  problem  and  much  work  remains  to  be  done. 
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